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Abstract. Since typical evolutionary design systems encode only a sin- 
gle artifact with each individual, each time the objective changes a new- 
set of individuals must be evolved. When this objective varies in a way 
that can be parameterized, a more general method is to use a represen- 
tation in which a single individual encodes an entire class of artifacts. In 
addition to saving time by preventing the need for multiple evolutionary 
runs, the evolution of parameter-controlled designs can create families of 
artifacts with the same style and a reuse of parts between members of the 
family. In this paper an evolutionary design system is described which 
uses a generative representation to encode families of designs. Because a 
generative representation is an algorithmic encoding of a design, its input 
parameters are a way to control aspects of the design it generates. By 
evaluating individuals multiple times with different input parameters the 
evolutionary design system creates individuals in which the input param- 
eter controls specific aspects of a design. This system is demonstrated 
on two design substrates: neural-networks which solve the 3/5/7-parity 
problem and three-dimensional tables of varying heights. 


1 Introduction 

Evolutionary algorithms have been used in a variety of different design domains, 
with each individual in the evolutionary design system typically encoding a single 
design. With this type of representation, each time the objective changes (such 
as the desired lift of an aircraft wing or the receptive properties of an antenna) 
a new evolutionary run must be performed. While one option is to use previous 
results to seed a new run - as Gruau did in evolving parity networks [1] - these 
additional evolutionary runs can be avoided by evolving individuals which take 
an input parameter that controls some feature of the resulting design. 

One method for an individual to encode a family of designs is for each mem- 
ber of the family to be encoded separately in the genotype. Yet with such a 
representation the size of the genotype grows with the number and size of each 
family member and it does not easily generalize to produce a design not already 
encoded. The alternative is to encode a family of designs with an algorithm 
which reuses parts of the genotype for different family members. 

In addition to efficiencies of space, the reuse of genotypic elements for multiple 
members in the design family has two other advantages. For consumer products 



it is often desirable to have different versions of a product that vary in some 
way, yet have the same style. Whereas the individuals produced by different 
evolutionary runs usually have a different structure to them, a single individual 
that generates a family of designs with a reuse of assemblies of components 
will produce designs with a similar style. This reuse of parts leads to a second 
advantage of evolving design families, which is improved manufacturability. Since 
the members of these design families have more components in common than 
designs produced by multiple runs, there are fewer different parts to test and 
having assemblies of parts in common across the entire family should result in 
lower manufacturing costs. 

An algorithm for encoding families of designs can be described as a program 
for mapping a seed to a design. Using these definitions existing work in evolution- 
ary design can be classified as evolving either a single seed, a program, or both 
together. For example, the evolution of a vector of parameters with Dawkins’ 
Biomorphs [2] and Ventrella’s stick creatures [3] is the evolution of a seed for 
a pre-defined creature-building program. More common is the evolution of pro- 
grams for fixed seeds, such as the evolution of cellular automata rules for a fixed 
starting state [4, 5] and the evolution of Lindenmayer systems (L-systems) with a 
fixed axiom [6, 7]. Finally, both seeds and programs have been evolved together, 
such as Frazer’s evolution of both starting condition and cellular-automata-style 
rules [8] and the evolution of the axiom and rules of L-systems by Jacob [9] and 
Hornby [10]. 

Previously we defined generative representations as the class of representa- 
tions in which elements of the genotype are reused in the translation to the 
phenotype and demonstrated a generative representation in which the genotype 
contained both a program for creating designs and the input parameters for the 
starting rule [10]. Here we describe an extension of this work from evolving a 
single design, with reuse of modules within the design, to evolving design fami- 
lies, with a reuse of modules across different members of the family. To produce 
individuals which represent a family of designs we now encode in the genotype 
only the program for constructing a design, and then evaluate an individual 
multiple times by compiling the program with different starting parameters. For 
each of these evaluations a specific phenotypic property of the resulting design 
is then compared with the desired result and the individual’s fitness is adjusted 
based on how closely they match. By testing an individual with different starting 
parameters in this way, individuals are evolved to be responsive to the parame- 
ter values. We demonstrate the generality of this approach by evolving families 
of designs on two different design substrates: neural-networks which correctly 
calculate 3/5/7-parity, and three-dimensional tables of varying height. 

The rest of this paper is organized as follows. First the generative representa- 
tion for encoding families of designs is described, followed by a description of the 
overall evolutionary design system. The next two sections describe the evolution 
of a family of networks for calculating the 3/5/7 odd-parity function and the 
evolution of three-dimensional tables of varying heights. Finally we close with a 
summary of this paper. 



2 L-systems as a Generative Representation 

The generative representation for our design families is based on parametric Lin- 
denmayer systems (PL-systems) [11]. PL-systems are a grammar consisting of 
a set of production rules for string rewriting. Production rules axe composed 
of a predecessor, which is the symbol to be replaced, followed by a number of 
condition-successor pairs. The condition is a boolean expression on the parame- 
ters to the production-rule, and the successor consists of a sequence of characters 
that replace the predecessor. For example in the production; 

; 712 > 5 —)■ ^(h 2 + l)oZ)(7i2 FO.5, Til — 2) 

the predecessor is A(ni, 1 x 2 ), the condition is 712 > 5 and the successor is B(ti 2+1) 
c D(n2+0.5, ni-2). Predecessor symbols are re-written by testing each of their 
conditions sequentially and then replacing the predecessor symbol with the suc- 
cessor of the first condition that succeeds. 

To generate strings with the grammar a starting string is required. For ex- 
ample the following grammatical rules, 

A(ni,7i2) : (tii > 0) a{rii) B(7i2,7ii) A(ni - 1 , 712 ) 

A(ni,7i2) : (tii < 0) -> a(0) 

B{ni,n 2 ) : (tii > 1 ) 6 ( 712 ) B{rii - 1 , 712 ) 

B {711,7x2) : {rii < 1) 6(712) 

when compiled staxting with the string, A(3,2), produce the sequence, 

A(3, 2) 

a{3)B{2,3)A{2,2) 
a(3)6(3)B(l, 3)a(2)B(2, 2)A{1, 2) 
a(3)6(3)6(3)a(2)6(2)B(l, 2)a(l)B(2, 1)A(0, 2) 
a(3)6(3)6(3)a(2)6(2)6(2)a(l)6(l)B(l,l)a(0) 
a(3)6(3)6(3)a(2)6(2)6(2)tt(l)6(l)6(l)a(0) 

The combination of A(3, 2) and the grammatical rules is a generative representa- 
tion for producing the final string in the sequence. In this case the seed consists 
of A{3, 2) and the program is the grammar. Alternatively, by using the starting 
string A(7ii,7i2), the grammar is a program for creating a family of designs: the 
first parameter, tii, controls the number of blocks of 6’s that axe created and the 
second parameter, ri 2 , controls how mainy 6’s axe in each block. 

By assigning a meaning to each symbol, the strings produced by a PL-system 
can be used to construct artifacts. Consider the following PL-system: 

PO(ni) : Til > 1.0 [ Pl(rii * 1.5) ] up(l) forward(3) 

down(l) P0 (tii — 1) 

P1(tii) : Til > 1.0 -J' { [ forward(rii) ] left(l) }(4) 

ff this PL-system is started with the string P0(4), it produces the following 
sequence of strings, 




Fig. 1. Two example structures. 


L P0(4) 

2. [ Pl(6) J up(l) forward(3) down(l) P0(3) 

{ [ forward(d) ] left(l) }(4) ] up(l) forward(S) down(l) [ Pl(4^5) ] 
up(l) forward(3) down(l) P0(2) 

^' [ { [ forward(S) ] left(l) }(4) } up{l) forward(3) down(l) [ { [ for- 
ward(4>5) ] left(l) }(4) ] up(l) forward(3) down(l) [ PI (3) ] up(l) 
forward(3) down(l) P0(1) 

^- [ { [ forward(6) ] left(l) }(4) } up(l) forward(3) down(l) [ { [ for- 
ward(4-5) ] left(l) ](4) ] up(l) forward(3) down(l) [ { [ forward(3) ] 
left(l) } (4) ] up(l) forward(S) down(l) 

6. [ f forward(d) J left(l) [ forward(6) ] left(l) [ forward(d) ] left(l) 
[ forward(6) ] left(l) ] up(l) forward(3) down(l) [ f forward(4-5) ] 
left(l) [ forward(4’3) ] left(l) [ forward(4-5) J left(l) [ forward(4>5) J 
left(l) } up(l) forward(3) down(l) [ [ forward(3) ] left(l) [ forward(3) 
] left(l) [ forward(S) ] left(l) [ forward(3) ] left(l) ] up(l) forward(3) 
down(l) forward(3) 


By interpreting the final string as a sequence of commands to a LOGO-style 
turtle, this PL-system creates the tree in figure La. 

To encode families of designs with this generative representation, the starting 
string (seed) is set to the predecessor of the first production rule with variables 
for starting parameters instead of numerical values. Tn this example the starting 
string is PO(ni) and different values of ni will produce different trees: the tree 
in figure l.b is created from this system by starting it with rii equal to six. 



3 Method 


The system for evolving design families uses a canonical evolutionary algorithm 
(EA) with variation operators customized for the representation. A generational 
EA is used in which each individual encodes a design family using the genera- 
tive representation described in section 2. Parents are selected with stochastic 
remainder selection [12] based on rank, using exponential scaling [13]. To cre- 
ate new individuals, the mutation and recombination operators of Hornby [10] 
are applied with equal probability. Mutation modifies an individual by chang- 
ing one symbol with another, perturbing the parameter value of a symbol, 
adding/ deleting some symbols, or recombining an individual with itself. With re- 
combination, one parent is the main parent and it is modified by swapping some 
genetic material - either an entire rule, a single production body or substrings 
of a production body - with a second parent. 

To produce individuals which encode for families of designs, individuals are 
evolved such that the value(s) of the input parameter(s) controls a certain feature 
of a design in a specific way. Each individual is tested with a range of different 
input values and each design’s score is modified by how well the feature in the 
design matches the desired result. The following two sections will describe the 
application of this system for two design substrates. 

4 Evolution of Parameter- Controlled n-Parity Networks 

The first substrate for which families of designs are evolved is neural networks 
which calculate the odd-paxity function. The odd-n-parity function returns true 
if the number of true inputs is odd and returns false otherwise. This function 
is difficult because the correct output changes for every change of an input value. 
In addition, the even /odd-n-parity functions have become a standard benchmark 
function in genetic programming (GP) and past experiments have shown that 
GP does not solve the five-parity (or higher) problem without automatically 
defined functions [14]. 

The method for using generative representations to encode neural networks 
is the same as our earlier work [15], which we now summaxize. First the genera- 
tive representation (section 2) is compiled into an assembly procedure and each 
neural network is constructed from an initial graph by executing this assembly 
procedure. The initial graph consists of a single neuron which has a single edge 
from itself to itself and the assembly procedure is a sequence of commands from 
the following command set, for which the current link connects from neuron A 
to neuron B: 

- add- input (n), creates an input neuron with a link from it to neuron B with 
weight n. 

- add-output(n), creates an output neuron with a link from B to it with 
weight n. 

- decrease-weight(n), subtracts n from the weight of the current link. If the 
current link is a virtual link, it creates it with weight — n. 


— duplicate(n), creates a new link from neuron A to neuron B with weight 
n. 

— increase-weight (n), adds n to the weight of the current link. K the current 
link is a virtual link, it creates it with weight n. 

— loop(n), creates a new link from neuron B to itself with weight n. 

— merge(n), merges neuron A into neuron B by copying ail inputs of A as 
inputs to B and replacing all occurrences of neuron A as an input with 
neuron B. The current link then becomes the nth input into neuron B. 

— next(n), changes the from-neuron in the current link to its nth sibling. 

— output (n), creates an output-neuron, with a linear transfer function, from 
the current from-neuron with weight n. The current-link continues to be 
from neuron A to neuron B. 

— parent (n), changes the from-neuron in the current link to the nth input- 
neuron of the current from-neuron. Often there will not be an actual link 
between the new from-neuron and to-neuron, in which case a virtual link of 
weight 0 is used. 

— [, pops the top state off the stack and makes it the current state. 

] , pushes the current state to the stack. 

— reverse, deletes the current link and replaces it with a link from 5 to A 
with the same weight as the original. 

— set-function(n), changes the transfer function of the to-neuron in the cur- 
rent link, B, with: 0, for sigmoid; 1, linear; and 2, for oscillator. 

— split (n), creates a new neuron, C7, with a sigmoid transfer function, and 
moves the current link from C to B and creates a new link connecting from 
neuron A to neuron C with weight n. 

The design problem is to evolve an individual that specifies a family of net- 
works with three/five/seven inputs which calculates the three /five/ seven-odd- 
paxity problem. To specify which network to construct, the first input parameter 
is set to 3.0, 5.0 and 7.0 to solve the three, five and seven parity problem. Input 
values are 1.0 for true and -1.0 for false. Networks are updated four times 
and then the value of the output neuron is examined to determine the parity 
value calculated for that set of input values. If the value of the output neuron 
is > 0.9 then the output of the network is taken as true and if the value of the 
output neuron is < 0.9 then the output of the network is taken as false. If the 
absolute value of the output neuron is < 0.9, the network is iteratively updated 
until its output value is either > 0.9 or < —0.9, for a maximum of six updates. 
The network receives a score of 2.0 for returning the correct parity value and a 
score of -1 for an incorrect answer. If the absolute value of the output neuron is 
less than 0.9 after six network updates, the network receives a score of 1.0 if the 
value of the output neuron is positive and the parity was true or if the value of 
the output neuron is negative and the parity is false. No penalty is given for 
having an incorrect value in this case. The fitness value of a network is the sum 
of its scores on all possible inputs and an individual’s fitness score is the sum of 
its scores for the three networks. 

Using the fitness function described in the previous paragraph, the graph 
in figure 2 contains a plot of the fitness of the best individual in the popu- 
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Fig. 2. Best fitness and average population fitness, averaged over fifty trials, for solving 
3/5/7-paxity. The maximum possible score is 336. 


lation averaged over fifty trials. For these trials the generative representation 
was set to a maximum of fifteen productions, each with two parameters and 
three sets of condition-successor pairs. The maximum number of commands in 
each condition-successor pair is fifteen and the maximum length of an assembly 
procedure generated by the representation is ten thousand commands. The evo- 
lutionary algorithm used a population of five hundred individuals and was run 
for five thousand generations. Out of these fifty runs, the generative representa- 
tion found a solution that produced correct 3/5/7-parity networks twelve times. 
For those runs that were successful it took 1800 generations, on average, to find 
a solution that produced correct networks. The smallest networks produced by a 
single individual that correctly calculates the 3/5/7-paxity problems are shown 
in figure 3 (the genotype of this individual is listed in Appendix B of [10]). 

5 Evolution of Parameter- Controlled Table Designs 

The second design problem is that of evolving families of tables in which the input 
parameter controls the height of the table. With this substrate, the command set 
consists of commands for controlling a LOGO-style turtle in a three-dimensional 
grid [16]. As the turtle moves it fills in voxels, creating a three-dimensional 
object. The commands for this substrate are: 

- back(n), move in the turtle’s negative X direction n units. 

- clockwise(n), rotate heading n x 90° about the turtle’s X axis. 

- counter-clockwise(n), rotate heading n x —90° about the turtle’s X axis. 

- down(n), rotate heading n x —90° about the turtle’s Z axis. 

- forward(n),move in the turtle’s positive X direction n units. 

- left(n), rotate heading n x 90° about the turtle’s Y axis. 

- [, pops the top state off the stack and makes it the current state. 
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Fig. 3. Networks constructed to solve 3, 5, 7-parity from the same evolved network 
encoding. 


- ] j pushes the current state to the stack. 

- right (n), rotate heading n x —90° about the turtle’s Y axis. 

- up(n), rotate heading n x 90° about the turtle’s Z axis. 

Section 2 contains an example of a design family encoded using this command 
set and the assembly procedure it compiles into. 

Rather than have the input parameter exactly specify the height of the table, 
we evolve tables whose height is four times higher than the value of the input 
parameter. Using a volume of 40x40x40 voxels, the maximum height of a table 
is forty units, so the valid range of input parameters is from one to ten. To al- 
low us to later determine if an evolved generative representation will interpolate 
between tested input values and extrapolate beyond tested input values, we eval- 


I 



uate an individual using four input values that cover the range: 2.0, 4.0, 6.0, 
and 8.0. As with the previous set of experiments, individuals are encoded with 
the generative representation of section 2 using a maximum of fifteen production 
rules, each with two parameters and three condition-successor pairs. Since pro- 
duction rules take two input parameters, the second input value is an evolved 
value and is fixed for all trials. 

To score the sensitivity of an individual to its input parameters, an individ- 
ual’s fitness is a combination of scores from the four designs created using four 
different input values. The fitness score for a single table design is based on that 
of our earlier work [16], with the objectives of maximizing stability and surface 
area while minimizing the amount of material used: 

f surf ace = the number of voxels at Ymax 
f Stability = ^ ^rea of the convex hull at height y 

y=0 

f material — number of voxels not on the surface 
The overall score of a single table combines these objectives into a single function: 

SCOTe(tClble) = /surface ^ f stability / /material (1) 

In addition, there is a height objective specified by the seed parameter: 

{ Ymax ! desired ^^Ymax ^ 

Ymax Ymax — height desired 

height desired I Ymax ff Ymax ^ heightdesired 


This height objective is a value in the range of zero to one that penalizes a 
design for under-shooting or overshooting the desired height. A single fitness 
value for an individual is created by summing the scores for each of the four 
tables created by the four different seeds and multiplying them by the sum of 
the height penalties for all four tables: 


/itness = 


^ / height 


^1=1 


X f ^ score (taWCx) 




( 2 ) 


The reason for summing all the height penalties and applying them to the scores 
for all tables is to put pressure on the EA to evolve individuals which are sensitive 
to the seed parameter. With an early version of this test function in which the 
height penalty for a given table was applied only to that table. 


/itness = /heightii^O'blej) x score{tablei) j (3) 

evolved individuals tended to produce tables with high fitness for some of the 
seed parameters and had low fitnesses for others. 
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Fig. 4. Best fitness and average population fitness, averaged over fifty trials, for the 
table design problem. 


Using the fitness function of equation 2, figure 4 contains a graph plotting the 
fitness of the best individual in the population and average fitness of the entire 
population averaged over fifty trials. Evolved tables were responsive to the seed 
values and in all fifty trials the individuals in the final generation created tables 
within one voxel of the desired height for all four seed values. Figure 5 contains 
six tables in the design family for one of the evolved parameter-controlled tables. 
The tables in 5.a-d are the tables that are generated with tested input parameters 
2.0, 4.0, 6.0 and 8.0 - the second parameter is evolvable and in this case is 2.0. 
The table in 5.e is an example with an input of 7.0, demonstrating that this 
individual can interpolate and generate a table with a seed value that is inside 
the tested range, and the table in 5.f is an example with an input of 10.0, which 
demonstrates that this individual can produce designs that extrapolate beyond 
the range tried during evolution. 

6 Summary 

Typical evolutionary design systems must evolve a new set of individuals each 
time the design objective changes. Here we presented an evolutionary design 
system in which individuals use a generative representation to encode a family 
of designs. By encoding designs as a program and not directly, the generative 
representation uses an input parameter to control a design feature. Individuals 
are evolved to be sensitive to this input parameter by evaluating each one mul- 
tiple times with different input values and combining the scores for the resulting 
designs into a single fitness function. 

Using this system, families of designs were evolved on two different problem 
domains. On the first design problem, individuals were evolved that encoded 
three networks that calculated 3, 5 and 7 parity. On the second domain, individ- 
uals were evolved such that an input parameter controlled the height of a table. 





(a) table(2.0, 2.0) (b) table(4.0, 2.0) 



(e) table(7.0, 2.0) (f) table(10.0, 2.0) 


Fig. 5. Parameter-controlled tables: (a)-(d) are the four trial parameters, (e) is an 
interpolation example, and (f) is an extrapolation example. 



In addition, it was demonstrated that one evolved individual produces tables of 
the correct height for an input value in between those tested during evolution 
and for an input value greater than those tested during evolution: examples of 
interpolation and extrapolation. In general, evolving families of designs with a 
generative representation produced individuals with a reuse of modules among 
members of the design family. This reuse produced designs in a similar style and 
should lead to improved manufacturability. 
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